Bayes-risk Consistency of Boosting
نویسندگان
چکیده
The probability of error of classification methods based on convex combinations of simple base classifiers by “boosting” algorithms is investigated. We show in this talk that certain regularized boosting algorithms provide Bayes-risk consistent classifiers under the only assumption that the Bayes classifier may be approximated by a convex combination of the base classifiers. Non-asymptotic distribution-free bounds are also developed which offer interesting new insight into how boosting works and help explain their success in practical classification problems.
منابع مشابه
Boosting method for local learning
We propose a local boosting method in classification problems borrowing from an idea of the local likelihood method. The proposed method includes a simple device to localization for computational feasibility. We proved the Bayes risk consistency of the local boosting in the framework of PAC learning. Inspection of the proof provides a useful viewpoint for comparing the ordinary boosting and the...
متن کاملBoosting Learning Algorithm for Pattern Recognition and Beyond
This paper discusses recent developments for pattern recognition focusing on boosting approach in machine learning. The statistical properties such as Bayes risk consistency for several loss functions are discussed in a probabilistic framework. There are a number of loss functions proposed for different purposes and targets. A unified derivation is given by a generator function U which naturall...
متن کاملVariable margin losses for classifier design
The problem of controlling the margin of a classifier is studied. A detailed analytical study is presented on how properties of the classification risk, such as its optimal link and minimum risk functions, are related to the shape of the loss, and its margin enforcing properties. It is shown that for a class of risks, denoted canonical risks, asymptotic Bayes consistency is compatible with simp...
متن کاملCombining Active Learning and Boosting for Naïve Bayes Text Classifiers
This paper presents a variant of the AdaBoost algorithm for boosting Näıve Bayes text classifier, called AdaBUS, which combines active learning with boosting algorithm. Boosting has been evaluated to effectively improve the accuracy of machine-learning based classifiers. However, Näıve Bayes classifier, which is remarkably successful in practice for text classification problems, is known not to...
متن کاملDiscussion of Boosting Papers
We congratulate the authors for their interesting papers on boosting and related topics. Jiang deals with the asymptotic consistency of Adaboost. Lugosi and Vayatis study the convex optimization of loss functions associated with boosting. Zhang studies the loss functions themselves. Their results imply that boosting-like methods can reasonably be expected to converge to Bayes classifiers under ...
متن کامل